A number of prior art devices have been developed to visually inspect objects so as to determine if they meet predefined criteria. Two basic types of vision inspection systems have been developed: those which use backlighting to silhouette the object and those which use frontlighting to observe the object as a variation of illumination across its surface area. The latter type of lighting produces an image, in black and white, known in the art as a gray scale since the observed image can vary in illumination throughout all shades of gray, including zero albedo (black) and 100 percent albedo (total reflectance).
Background illuminated vision systems provide for rather easy edge detection due to the rapid transition between illuminated regions outside of the object and non-illuminated silhouette regions obstructed by the object. When the edges are not so clearly distinguished, such as when an object is frontlighted, different techniques have been used on the gray scale image to develop an edge map. Such a map actually consists of a two-dimensional binary representation of the illumination level of the object. As set forth in a book entitled Basic Robotics Concepts by John M. Holland, Howard W. Sams and Company, 4300 W. 62nd Street, Indianapolis, Ind., pages 192-224, most methods for determining edges that use a gray scale image determine an edge map through use of a derivative and/or second derivative of the gray scale values in two dimensions. The use of differentiation to detect edges poses a noise problem since derivative signals are inherently susceptible to large transient values. As noted in the Holland text, E. C. Hildreth and D. Marr of the Massachusetts Institute of Technology have discovered that if the images are first smoothed through filtering at different spatial bandwidths, the resultant images when subjected to differential edge detection produce edge maps which can be correlated to find the edges of the real objects (see Hildreth, Helen C., Implementation of a Theory of Edge Detection, M.I.T. TR-579 and Marr, D. Vision, W. H. Freeman Company, San Francisco, Calif., 1981).
Another form of edge detection is known as the Sobel technique as described at page 337 of a book entitled Digital Image Processing by Gonzales and Wintz, published by Addison Wesley Company, 1977. For a 3.times.3 image defined by rows ##STR1## and the gradient at midpoint e is defined as EQU G=(G.sub.x.sup.2 +G.sub.y.sup.2).sup.1/2, (EQ 1-1)
or using absolute values EQU G=.vertline.G.sub.x .vertline.+.vertline.G.sub.y .vertline.;(EQ 1-2)
where EQU G.sub.x =(c+2f+i)-(a+2d+g) (EQ 1-3)
and EQU G.sub.y =(a+2b+c)-(g+2h+i) (EQ 1-4)
This technique for edge detection is used by the present invention. It has also been used by the General Motors Corporation in a vision inspection system called SIGHT-I, described in an article entitled "General Motors Inspects IC Chips" appearing in the May 1980 issue of Computer magazine, published by the Institute of Electrical and Electronics Engineers, Inc. (IEEE) Computer Society, Los Almitos, Calif. In the SIGHT-I system, the gradient information is only used to determine approximate rotation of an object under test and is not used as a general technique to compare two objects.
The SIGHT-I system also describes a technique for recognizing an object which is under test, in this particular case a small integrated circuit (IC) chip which has been bonded to a heat sink.
After the approximate orientation of the object is determined, the corners of the object are determined by the difference in illumination to be expected at corners; that is, the differences in illumination between darkness and lightness. Thus, for instance, the northwest corner of the chip should have white areas at its top and left sides. Four local illumination intensity templates are used to identify the characteristics of the four types of corners and are matched against each illumination pixel of the array using a weighting algorithm.
The actual corners are determined by the fixed relationship of the ob3ect since its actual shape is known. In the example with the integrated circuit chip, it is known that the corners comprise four 90.degree. angles, that the chip has specified dimensions and an approximate orientation angle. These characteristics are combined into a global template which is matched against sets of possible corners. An algorithm then calculates a "goodness of fit" figure for each set and selects the best fitting set as the actual corners.
Thus although gradient information is used to determine a probable orientation of the IC chip, illumination information is used to compare the corners of a known IC chip to potential corners of an IC chip under test. No gradient information is used to generate the corner templates nor to derive information about selected regions on the IC chip under test.
The present invention is distinguished from the GM technique since the present invention compares a known object to a test object by use of gradient information, not illumination information. By so doing, the system becomes immune to changes in ambient light or other paramenters that can affect absolute illumination values. The GM technique is not immune to such changes.
Furthermore, in the present invention, orientation of the object under test is determined by finding the angular and displacement differences between the selected points on the known object and the recognized points on the object under test. Overall gradient information of the object under test is not used, as in the GM system, to determine probable angular movement of the IC chip under test. Therefore the use of gradient information in the GM system is for a different purpose than its use in the present invention.
The present invention also provides means for using various visual tools for measuring or determining the presence of specified parameters or features on the object under test. Such parameters and features include a visual tool to measure the size of an object in a manner similar to that performed by mechanical calipers, a tool to determine an alphanumeric or other symbol displayed on an object through use of character segement recognition, a tool to locate the center of a hole within an object, a tool to measure the angle subtended by three points on an object, a tool to count the number of edges in a specified region as compared to a known object, and other tools which may be required for identification or measurement of an object typically in an industrial assembly or testing environment.
Therefore it is submitted that the present invention provides a new vision inspection system based upon the property of edges which allows for the rapid identification and orientation of an object under test as compared to a known object. By allowing the user to define the edge locations of an object for comparison to unknown objects, the system provides the flexiblility to meet the requirements of the user without being constrained by particular characteristics of the objects under test. Also, since only areas about selected locations are used in the recognition process, other areas of the object under test can vary without affecting this process. This feature of the present invention is valuable in inspection applications where only certain features of the object under test need correspond to a known object. Indeed such objects may not be subject to testing using other vision inspection techniques.
The present invention thus differs from other pattern recognition techniques including template matching in which the entire image of an object is stored in memory for comparison to images seen by the camera (see pages 205-207 of Basic Robotics Concepts, cited above). It also differs from techniques which are tolerant to orientation variations between a known object and an object under test such as those that use area, moments, and axes of minimum inertia as described in Basic Robotics Concepts, above, at pages 205-212, or through use of a technique known as moment invariants as described in Basic Robotics Concepts at pages 212-213 and the appendices thereto.
In distinction from pattern recognition techniques that use moments or moment invariants to determine a correspondence between two objects, the present invention uses points selected by the user to define those areas on a known object which characteristically represent that object. Such points are visually perceptible to the user, unlike the highly abstract nature of moments and moment invariants which have not direct visual representation.
Furthermore, by allowing points to be selected by the user on the known object, other portions of the object which may change in size or shape, can be ignored for purposes of recognition testing.
Indeed, the present invention need not know any particular information concerning the overall shape of an object provided that the object of interest can be defined by distinct edges. In many testing applications of component parts, this is all that need be performed in order to find and determine the orientation of an object under test. In most applications, however, the visual tools of the present invention are used once the object is identified and oriented. These tools provide valuable information required to perform automatic inspection. Such visual tools are a unique feature of the present invention and allow it to easily adapt to the particular needs of the user.
These visual tools and positioning locations provide a working environment that is familiar to people who traditionally use mechanical devices to measure and test objects. Thus the three positioning locations typically selected on the known object correspond to mechanical positioning pins used to orient an item to be tested. The visual tools of the present invention likewise correspond to such measuring instruments, such as calipers, used to mechanically measure objects.
It is submitted that the present invention provides a new technique and system for performing visual inspection of objects.